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Abstract 


As  the  Department  of  Defense  (DoD)  moves  from  the  single-system,  platform¬ 
centric  paradigm  to  the  capabilities-based  paradigm,  the  scope  and  complexity  of 
solutions  are  growing.  The  increasing  emphasis  on  joint  service  and  system-of- 
systems  (SoS)  capabilities  has  created  both  opportunities  and  challenges  for 
materiel  acquisition.  A  key  barrier  that  needs  to  be  overcome  for  the  DoD  to  achieve 
the  promises  of  joint  service  and  SoS  programs  involves  the  challenge  of 
“transaction  costs.”  These  are  the  less  visible,  but  nonetheless  significant,  costs  of 
negotiating,  managing  and  monitoring  transactions.  In  an  effort  to  identify  the  effect 
of  transaction  costs  on  more  complex  acquisition  programs,  this  paper  examines 
cost  and  schedule  breaches  in  a  subset  of  Major  Defense  Acquisition  Programs 
(MDAP)  that  includes  a  sample  of  84  programs,  divided  into  “joint  service”  and 
“traditional”  (single  service)  acquisition  programs,  and  “single  system”  and  “system- 
of-systems”  (SoS)  programs.  The  results  suggest  there  is  a  statistically  significant 
higher  risk  of  cost  and  schedule  breaches  in  SoS  programs  than  in  single  system 
acquisition  programs.  This  paper  contributes  to  a  broader  study  that  eventually 
needs  to  be  conducted  that  will  evaluate  the  benefits  and  costs  of  increased  reliance 
on  joint  service  and  SoS  programs. 

Keywords:  Transaction  Costs,  Acquisition,  Joint  Service,  System-of- 
Systems,  Cost  Breaches,  Schedule  Breaches,  Cost  Estimating 
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1 


Introduction 


The  US  Department  of  Defense  (DoD)  is  in  the  process  of  radical 
transformation — transformation  to  a  national  security  strategy  predicated  on  joint 
service  purchases  and  complex  system-of-systems  (SoS)  capabilities.1  The  DoD’s 
increasing  emphasis  on  joint  service  and  SoS  capabilities  has  created  both 
opportunities  and  challenges  for  materiel  acquisition.  In  terms  of  improving  the 
effectiveness  of  warfighting  capabilities,  the  opportunity  exists  for  joint, 
interoperable,  multi-function,  multi-mission  systems  that  leverage  information 
dominance  and  improve  decisions  and  outcomes  by  making  US  and  coalition  forces 
not  only  better  informed  but  also  more  coordinated,  faster  and  more  adaptive.  In 
terms  of  efficiencies,  multiple  opportunities  exist  for  joint  programs  to  cut  “economic 
production  costs” — by  reducing  duplication,  by  exploiting  learning  curves,  and  by 
achieving  economies  of  scale  and  scope  in  manufacturing  and  in  operations  and 
support  activities  (e.g.,  joint  training  and  logistics). 

As  we  move  from  the  single-system,  platform-centric  paradigm  to  the 
capabilities-based  paradigm,  the  scope  and  complexity  of  solutions  are  growing.  As 
we  move  beyond  our  experience  base  (and  our  empirical  data  sets)  we  must  ask 
some  profound  questions: 

■  As  the  magnitude  and  complexity  of  DoD  systems  increase,  how  do  we 
develop  realistic  cost,  schedule,  &  performance  baselines? 

■  Are  we  measuring  the  right  drivers  of  cost/schedule? 

■  Are  linear  assumptions  and  models  valid? 


1  As  defined  in  the  DoD  Defense  Acquisition  Guidebook  (2004),  an  SoS  is  “a  set  or  arrangement  of 
systems  that  results  when  independent  and  useful  systems  are  integrated  into  a  larger  system  that 
delivers  unique  capabilities.”  Joint  Program  Management  is  defined  as  “Any  defense  acquisition 
system  [. . .]  or  technology  program  that  involves  formal  management  or  funding  by  more  than  one 
DoD  Component  during  any  phase  of  a  system’s  life  cycle”  (DAU,  2004,  p.1).  DoD  Directive  5000. 1, 
The  Defense  Acquisition  System,  dated  12  May  2003,  indicates  a  policy  preference  for  joint 
development  programs  over  Component-unique  development  programs  (AT&L). 
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The  answers  to  these  questions  lie  at  the  heart  of  our  ability  to  provide 
reliable  estimates  on  projected  outcomes  for  major  defense  investments  to  senior 
defense  decision-makers  as  illustrated  in  Figure  1 . 


Size,  SLOC,  #  of  Users,  #  of  Interfaces, 
#  of  Systems,  etc. 


Figure  1.  The  New  Systems  Acquisition  Context 

A  key  barrier  needs  to  be  overcome  for  the  DoD  to  achieve  the  promises  of 
joint  service  SoS  programs.  This  involves  the  challenge  of  “transaction  (coordination 
and  motivation)  costs.”  These  are  the  less  visible,  but  nonetheless  significant,  costs 
of  negotiating,  managing  and  monitoring  transactions. 

This  paper  investigates  cost  and  schedule  breaches  in  a  subset  of  Major 
Defense  Acquisition  Programs  (MDAP)  that  includes  a  sample  of  84  programs, 
divided  into  “joint  service”  and  “traditional”  (single  service)  acquisition  programs,  and 
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“single  system”  and  “system-of-systems”  (SoS)  programs.  The  results  suggest  there 
is  a  statistically  significant  higher  risk  of  cost  and  schedule  breaches  in  SoS 
programs  than  in  “single  system”  acquisition  programs.  This  paper  contributes  to  a 
broader  study  that  eventually  needs  to  be  conducted  that  will  evaluate  the  benefits 
and  costs  of  increased  reliance  on  joint  service  and  SoS  programs. 


NAVAL  POSTGRADUATE  SCHOOL 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 


-3- 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


2.  Joint  Service  and  System-of-Sys terns  Programs 


In  2003,  the  Joint  Capabilities  Integration  and  Development  System  (JCIDS) 
replaced  the  “Requirements  Generation  System”  that  had  identified  warfighter  needs 
for  nearly  30  years  (recently  updated  in  CJCS,  2007,  May  1 ).  Providing  a  new, 
substantive  role  for  Combatant  Commanders,  the  JCIDS  process  reflects  a 
significant  shift  in  the  focus  of  defense  programming  toward  joint  system  capabilities. 

Greater  emphasis  on  joint  operations  in  the  DoD  has  led  to  the  requirement 
for  interoperability  not  only  between  new  systems  and  legacy  systems  but  also 
between  systems  developed  by  different  agencies  or  services.  This  requires 
cooperative  management  of  development  and  acquisition  programs.  Crossing 
management  domains  creates  or  intensifies  issues  related  to  decentralized 
management  and  independent  authority  of  service  agencies  and  can  result  in 
asymmetrical  incentives  for  SoS  versus  “single  system”  and/or  “single  service” 
weapon  systems. 

In  several  recent  cases,  the  size  and  complexity  of  undertaking  SoS 
programs  has  overwhelmed  the  DoD’s  ability  to  effectively  manage  them.  According 
to  a  recent  Congressional  Research  Service  report,  “management  and  oversight  of 
acquisition  programs  increases  as  the  value  of  the  program  increases”  (Chadwick, 
2007,  p.  12).  The  larger  the  program,  “the  more  difficult  it  is  to  sustain 
communications  among  staff  members.  In  general,  if  there  are  ‘n’  people  in  a 
program,  the  potential  number  of  pair-wise  channels  is  n(n-1)/2  [...Ljarger  teams  [...] 
have  a  greater  chance  of  communications  breakdown”  (DAU,  2004,  p.  16).  Ceteris 
paribus — the  bigger  the  program,  the  larger  the  transaction  (coordination)  costs. 

There  are  a  variety  of  categories  of  joint  service  programs.  These  range  from 
relatively  simple,  ’’single  system”,  “single  service”  programs  to  which  other  services 
sign  on  to  use  the  end  product,  to  fully  integrated,  multi-service  SoS  programs. 
Examples  of  the  latter  include  the  Joint  Strike  Fighter  (JSF)  and  the  Joint 
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Surveillance  Target  Attack  Radar  System  (JSTARS).  Clearly,  the  latter  (joint  service 
SoS)  acquisitions  are  considerably  more  challenging  than  the  former.  The  reason 
can  be  traced  to  greater  interdependence,  manifested  in  greater  complexity  and 
uncertainty. 

Most  weapon  system  programs  are  interdependent  in  one  way  or  another. 
They  often  exchange  resources  and  information,  which  imposes  additional  costs  and 
constraints  on  the  efficient  execution  of  the  program.  The  effects  of 
interdependence  and  complexity  can  be  largely  hidden  however,  since  the  majority 
of  weapon  systems  are  still  managed  in  a  single  service  environment.  In  essence, 
the  DoD  defines  capabilities  at  the  joint  level  but  manages  the  majority  of 
acquisitions  at  the  service  level.  As  a  result,  the  costs  of  interdependence  and 
complexity  are  not  routinely  captured  or  included  in  cost  and  budget  estimates. 

Joint  service  SoS  acquisitions  involve  more  stakeholders,  multiple  users  and 
funding  sources,  divergent  and  competing  requirements,  and  conflicting  objectives 
that  lead  to  difficult  and  contentious  trade-offs,  diffused  authority,  negotiated  budget 
arrangements,  complex  project  management  structures,  etc.  This  increased 
interdependence  is  generally  reflected  in  greater  transaction  costs:  higher 
“coordination  costs”  from  increased  complexity  and  uncertainty,  and  higher 
“motivation  costs”  from  greater  asset  specificity  and  limited  market  contestability. 
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3.  Transaction  Costs  =  Motivation  Costs  + 
Coordination  Costs 


In  business,  two  costs  are  typically  factored  into  strategic  acquisition 
decisions:  production  costs  and  the  costs  of  managing  the  transaction — or 
“transaction  costs”  (Coase,  1937).  Conventional  strategies  tend  to  focus  on 
economic  production  costs  (input  costs,  learning  curves,  economies  of  scale  and 
scope,  etc.).  TCE  emphasizes  another  set  of  costs — coordination  and  motivation 
costs.  Economic  production  (opportunity)  cost  advantages  tend  to  guide  companies 
to  specialize  in  “core”  activities  in  which  they  have  a  comparative  advantage  and  to 
“transact”  with  outside  suppliers  to  acquire  other  goods  and  services.  A  key 
contribution  of  TCE  is  to  formally  introduce  and  fully  reveal  the  nontrivial  costs  of 
managing  those  transactions. 

Transaction  costs  include  coordination  and  motivation  costs,  such  as  search 
and  information  costs;  decision,  contracting  and  incentive  costs;  and  measurement, 
monitoring  and  enforcement  costs.  TCE  predicts  these  costs  will  vary  across 
weapon  system  acquisition  programs  to  the  extent  there  are  differences  in  certain 
key  characteristics  of  the  transaction — complexity,  uncertainty,  frequency,  and  asset 
specificity — as  well  as  market  contestability,  and  the  choice  of  governance 
(contracting,  etc.)  mechanisms.  A  central  point  from  TCE  is  that  the  choice  of 
contract,  organization,  and  incentives,  along  with  key  characteristics  of  the 
transaction  (complexity,  uncertainty,  frequency,  and  asset  specificity)  and  market 
contestability,  must  be  considered  in  order  to  obtain  reliable  cost  estimates  of  joint 
programs  (Melese,  Franck,  Angelis  &  Dillard,  2007). 

One  of  the  key  insights  of  TCE  is  that  capital  (and  human  capital  investments) 
that  are  specific  to  a  transaction  (e.g.,  made  to  support  a  joint  program  acquisition) 
can  generate  cost  savings  but  also  carry  the  risk  of  increasing  transaction  costs  from 
opportunistic  behavior.  The  role  of  relation-specific  investments  (“asset  specificity”) 
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is  an  important  consideration  that  needs  to  be  anticipated  and  factored  into  any 
analysis  of  “joint  service”  and  SoS  programs.2 


2  Williamson  (1996)  identifies  six  types  of  asset  specificity:  1)  site,  2)  physical  asset,  3)  human  asset, 
4)  dedicated  asset,  5)  brand-name  capital,  and  6)  temporal.  These  are  specialized  investments  that 
generate  high  returns  within  a  specific  relationship  but  offer  little  value  outside  it.  Site  specificity  refers 
to  the  co-location  of  facilities  to  minimize  inventory  or  production  costs.  Physical  asset  specificity 
refers  to  the  use  of  customized  assets  such  as  specialized  dies  and  equipment.  Human  asset 
specificity  refers  to  firm-specific  knowledge  and  skills  (e.g.,  “specific”  as  opposed  to  “general” 
training).  Dedicated  asset  specificity  refers  to  additional  investments  in  plant  and  equipment  made  to 
sell  the  extra  output  to  a  specific  customer.  Brand-name  capital  specificity  refers  to  investments  in 
reputation.  Temporal  asset  specificity  refers  to  investments  that  facilitate  timing  and  coordination  of 
projects  (e.g.,  investments  in  critical-path  activities). 


NAVAL  POSTGRADUATE  SCHOOL 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 


-8- 


4.  Motivation  Costs:  The  Role  of  Asset  Specificity 


Having  made  a  specialized  investment  in  location,  physical,  human,  or  other 
specific  assets,  a  supplier  often  becomes  the  most  efficient  provider,  which  is  good 
from  a  production-cost  perspective,  but  provides  incentives  for  the  supplier  to  look 
for  opportunities  to  extract  more  from  the  transaction  (perhaps  by  demanding  steep 
prices  for  any  modifications  to  the  contract).  After  investments  in  specific  assets  are 
made,  the  relationship  is  transformed  from  a  customer  having  the  choice  of  a 
number  of  competing  suppliers  to  a  bilateral  monopolistic  relationship  between  a 
buyer  and  seller. 

Similar  to  “sunk  costs,”  investments  in  relationship-specific  assets  (“asset 
specificity”)  are  potentially  valuable  but  can  increase  risks  to  both  parties  in  a 
transaction  (Klein,  Crawford,  &  Alchian,  1978).  Close-in  bilateral  bargaining  (a 
principal-agent  type  game)  replaces  the  competitive  marketplace.  This  entails  a 
transformation  of  the  supplier  from  competitive  bidder  (prior  to  source  selection)  to 
monopoly  supplier  (after  source  selection),  especially  if  there  are  no  close 
substitutes.  Accordingly,  the  customer  (government)  is  now  vulnerable  to 
“opportunistic  behavior”  from  the  supplier. 

Unforeseen  contingencies,  combined  with  newly  inelastic  demand,  may 
prompt  the  supplier  to  extract  more  of  the  surplus  created  in  the  relationship.3  In  this 
case,  suppliers  can  exploit  their  power  in  the  relationship  by  renegotiating  a  basic 
agreement  to  their  advantage,  otherwise  threatening  to  dissolve  the  agreement.  The 
TCE  literature  refers  to  this  as  a  “hold-up”4  (Klein,  Crawford,  &  Alchian  1978). 


3  Williamson  (1975),  Besanko,  Dranove,  Shanley  &  Schaefer  (2000),  and  others  have  labeled  the 
transition  from  one  prospective  buyer  and  many  sellers  to  one  buyer  and  one  seller  or  from 
competitive  market  to  bilateral  monopoly,  as  the  “fundamental  transformation.”  To  some  extent,  this 
transformation  occurs  after  the  completion  of  every  military  source-selection  process. 

4  According  to  Besanko  et  al.  (2000),  a  holdup  problem  arises  when  a  party  in  a  contractual 
arrangement  exploits  the  other  party’s  vulnerability  due  to  relationship-specific  assets. 
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Conversely,  a  supplier  (defense  contractor)  that  makes  specific  investments 
in  assets  that  are  only  valuable  in  the  context  of  the  relationship  with  a  specific 
customer  (government),  can  find  itself  vulnerable  to  any  changes  in  demand  from 
that  customer  (i.e. ,  the  supplier  suffers  from  “demand  uncertainty”).  Given  the 
government  is  the  only  buyer  (or  one  of  only  a  few)  of  joint  SoS  weapon  systems, 
and  given  its  limited  ability  to  commit  as  a  result  of  the  annual  nature  of  most 
budgetary  processes,  defense  industry  sellers  often  face  a  monopsony — a  buyer 
that  cannot  make  credible  multi-year  commitments.  This  leads  to  sellers  facing 
substantial  demand  uncertainty  and  the  real  risk  of  strategic  renegotiation. 

Whereas  relation-specific  investments  can  increase  the  total  gains  to  both 
parties,  the  risk  exists  of  opportunistic  behavior;  either  party  can  hold  up  the  other, 
for  instance,  by  threatening  to  change  the  terms  of  the  contract  (e.g.,  the 
government’s  sovereign  right  to  terminate  a  contract  for  convenience  as  well  as 
default).  The  danger  is  that  if  neither  party  feels  it  can  recover  the  full  costs  of  its 
investment  in  the  relationship/transaction  (say  through  a  continuation  or  renewal  of 
the  contract — “frequency”),5  then  efficiency-generating,  specific  investments  will  not 
be  made,  resulting  in  higher  costs.6 

It  is  important  to  note  at  this  point  that  whereas  TCE  has  traditionally 
examined  the  customer-supplier  relationship  in  the  context  of  a  contractual 


5  In  terms  of  “frequency,”  past  experience  with  similar  programs  appears  to  have  a  significant  impact 
on  a  supplier’s  costs  and  capabilities.  So,  if  source  selection  and  strategic  partnership  decisions 
recognize  this  and  clearly  favor  past  performance,  the  acquisition  process  will  be  converted  from  a 
one-shot  game  into  a  repeated  game,  allowing  suppliers  to  earn  a  return  on  their  investment  in 
reputation.  In  this  way,  increasing  frequency  through  strategic  partnerships  and  recurrent  transactions 
can  mitigate  opportunistic  behavior  and  build  trust  in  the  contracting  relationship.  By  identifying  key 
characteristics  of  transactions  such  as  frequency  and  fully  understanding  their  implications,  decision¬ 
makers  could  mitigate  cost,  schedule  and  performance  breaches. 

6  Scope  for  opportunistic  behavior  may  lead  to  adverse  selection,  choice  of  an  (ex-ante)  inferior 
option  (or  technology),  or  moral  hazard.  Such  scope  increases  risks  that  if  a  relationship-specific 
investment  is  made,  the  other  party  will  exploit  the  terms  of  the  contract  to  “hold  them  up.”  For 
example,  changes  in  specifications  are  frequently  used  by  contractors  as  a  reason  to  raise  prices  and 
profits  under  government  contracts — especially  when  those  investments  by  the  contractor  create 
barriers  to  the  entry  of  competitors. 
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arrangement,  the  domain  of  joint  capabilities  acquisition  is  distinctive  (though  not 
exclusively  so)  in  the  establishment  of  partnerships  among  government  entities 
(such  as  between  DoD  services  or  agencies),  as  well  as  teaming  arrangements 
among  private-sector  enterprises  within  the  defense  industrial  base  (e.g.,  product 
development  contractors  and  their  first-  and  second-tier  suppliers).  These  internal 
relationships  also  incur  transaction  costs  related  to  coordination  and  motivation 
costs;  search  and  information  costs;  decision,  contracting  and  incentive  costs,  etc. 
This  notion  is  explored  further  below. 

Whereas  defense  acquisition  has  traditionally  focused  on  economic 
production  costs,  joint  programs  expose  the  DoD  to  the  potentially  greater  costs  of 
managing  the  relationship,  and  more  importantly,  to  the  risks  of  opportunistic 
behavior  on  the  part  of  contracting  partners — a  critical  “transaction  cost.”  Given  the 
multiple  competing  stakeholders  in  a  joint  SoS  acquisition,  the  principle  of  self- 
interest  suggests  all  sides  have  incentives  to  behave  opportunistically  and  may  not 
necessarily  have  the  motivation  to  cooperate  to  make  cost-saving  investments — 
particularly  when  specific  assets  are  involved  and  information  is  imperfect 
(incomplete  or  uncertain)  and  asymmetric.7  Clarification  of  the  risks  of  “opportunism” 
(i.e.,  motivation  costs)  is  one  of  the  key  advantages  of  TCE. 

If  it  turns  out  that  joint  service  SoS  programs  require  a  significantly  greater 
ratio  of  specific  assets  to  total  investments,  then  this  increases  the  risk  of  bilateral 
dependency  and  “hold-up.”  Moreover,  given  the  difficulty  of  writing  complete 
contracts  for  joint  SoS  programs  that  will  cover  every  contingency,  with  incomplete 
contracts  the  hold-up  problem  poses  additional  risks  for  the  government,  such  as 


7  In  order  to  combat  this  tendency,  and  in  the  spirit  of  resolving  the  principal-agent  problem,  an 
interesting  incentive  clause  is  included  by  the  US  Air  Force  in  its  “National  Polar-Orbiting  Operational 
Environmental  Satellite  System.”  When  establishing  top  executives’  salaries  and  bonuses,  the 
contract  requires  TRW’s  corporate  board  to  consider  contract  performance.  By  tying  senior  executive 
pay  directly  to  contract  performance,  decision-makers  can  help  align  incentives,  increase 
accountability  and  reduce  cost  overruns  (Graham,  2003). 
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contractors  charging  excessively  high  prices  for  change  orders  and  strategic 
renegotiations. 
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5 


Governance  Issues 


TCE  suggests  the  degree  of  completeness  of  a  contract  is  an  optimizing 
decision  by  the  parties  involved,  one  that  reflects  tradeoffs  between  ex-ante 
investments  in  contract  design  and  the  risk  of  ex-post  costs  of  opportunistic 
renegotiation.  In  reality,  contracting  offers  an  imperfect  solution  to  opportunism. 

What  may  be  required  are  additional  governance  mechanisms  (i.e.,  rules  and 
regulations,  reputation  mechanisms,  termination  agreements,  government-furnished 
equipment,  Government  Owned  Contractor  Operated  (GOCO)  facilities,  warranties, 
etc.)  to  shift  risks  to  safeguard  and  protect  transaction-specific  investments,  settle 
disputes,  and  adapt  to  new  conditions.  Ex-ante  efforts  may  also  be  necessary  to 
screen  for  reliability  and  reputation  (e.g.,  pre-award  contract  surveys  of  potential 
venders).  These  structures  can  include  anything  from  agreements  to  share  and 
verify  cost  and  performance  information  through  incentive  contracts,  to  the  careful 
crafting  of  dispute-settlement  mechanisms  (e.g.,  alternate  dispute  resolution, 
proactive  management  councils,  etc.).  Among  government  entities  in  joint  acquisition 
programs,  memoranda  of  understanding  or  agreement  (MOUs  or  MOAs)8  reflect  a 
“quasi  contractual”  relationship  among  ostensibly  sovereign  entities.  The 
enforceability  of  these  is  always  questionable,  but  they  nevertheless  serve  to  make 
the  particulars  of  agreements  among  the  parties  explicit  and  provide  both  implicit 
and  explicit  dispute-resolution  mechanisms  to  reduce  the  risk  of  hold-up. 

In  general,  the  less  complex  and  uncertain  a  transaction,  and  the  lower  the 
requirement  for  specific  assets,  the  easier  it  is  to  write  an  explicit  contract  that 
covers  all  contingencies.  Also,  the  lower  the  administrative  and  enforcement  costs  of 


8  The  Air  Force  F-35  Joint  Strike  Fighter  (JSF)  program  featured  elaborate  memoranda  of  agreement 
among  domestic  and  international  partners  to  reduce  the  risk  of  hold-up.  Despite  these  structures, 
the  sovereignty  of  partner  governments  limited  the  enforceability  of  these  instruments,  and  hold-ups 
did  occur — notably  when  the  legislatures  of  the  partner  governments  imposed  changes  to  the 
agreements  articulated  in  the  MOAs,  upon  which  program  plans  were  based.  The  consequential 
impact  of  program  cost,  schedule,  and  performance  outcomes  have  not  been  fully  characterized  but 
would  be  a  worthwhile  subject  of  future  TCE  research. 
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that  contract,  the  lower  the  risk  of  hold-up.  These  favorable  characteristics  are  more 
likely  to  exist  in  established,  traditional  (“single  service”),  “single  system”  acquisition 
programs,  and  contribute  to  lower  costs  (or  cost  overruns),  and  better  performance 
and  schedules.9 

Evidence  uncovered  by  Bajari  and  Tadelis  (2001)  in  construction  contracts 
reveals  that  in  cases  in  which  a  transaction  is  easy  to  define  and  measure  (i.e.,  there 
is  little  complexity)  and  only  a  few  minor  changes  are  expected  (i.e.,  there  is  little 
uncertainty),  fixed-price  type  contracts  tend  to  dominate.  However,  the  more 
complex  the  transaction — the  more  difficult/costly  it  is  to  define  and  measure 
performance;  the  more  uncertain  the  transaction — the  more  likely  it  is  that  a  change 
in  the  contract  will  be  required  and  the  more  severe  the  adversarial  relationships 
experienced  ex-post  when  fixed-price  contracts  are  chosen.  In  the  latter  case,  fixed- 
price  type  construction  contracts  often  end  in  costly  renegotiations — in  which  any 
surplus  generated  was  dissipated  in  the  course  of  those  negotiations  through 
unproductive  bargaining  and  influence  activities.  Thus,  complexity  and  uncertainty 
can  force  parties  to  turn  away  from  fixed-price  type  contracts  and  towards  cost- 
reimbursement  type  contracts  (e.g.,  costs  plus  a  award/incentive  fee)  and  to  rely 
heavily  on  reputation  and  other  enforcement  mechanisms  to  avoid  ex-post 


9  If  a  transaction  requires  little  in  the  way  of  specific  assets  (no  hold-up  problem),  and  involves  a 
product  or  service  that  is:  a)  well-defined  and  homogeneous,  b)  easy  to  measure  (limited  complexity 
and  mild  information  asymmetry),  c)  routinely  used  (recurring/frequent  purchases),  d)  not  subject  to 
change  (limited  demand  uncertainty),  and  e)  is  offered  by  competing  suppliers,  then  there  is  little 
room  for  negotiation  (price  and  performance  are  market-driven),  and  the  marginal  benefit  of 
unproductive  bargaining  is  near  zero.  With  little  room  for  bargaining  over  such  routine  and 
uncomplicated  transactions,  substantial  production  and  transaction  cost  stability  can  be  expected  in 
the  acquisition.  Moreover,  since  administrative,  incentive,  and  enforcement  costs  tend  to  be  low  for 
acquisitions  in  more  contestable  (competitive)  markets,  the  marginal  cost  of  engaging  in  the 
transaction  is  relatively  smaller  for  the  military,  and  there  exists  an  incentive  for  the  supplier  to  invest 
in  the  transaction  that  generates  opportunities  for  cost  savings.  International  competition  for  standard 
(off-the-shelf)  commercial  components  of  weapon  systems  might  be  an  example.  By  unbundling 
large,  complex  weapons  systems  into  sub-systems,  decision-makers  might  reveal  opportunities  to 
enjoy  the  benefits  of  lower  transaction  costs  and  greater  competition,  leading  to  lower  production 
costs.  These  favorable  characteristics  generally  lend  themselves  to  more  accurate  cost  estimating 
and  may  result  in  fewer  cost  breaches. 
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opportunistic  behavior  that  threatens  to  dissipate  the  gains  generated  by  a 
transaction. 

In  reality,  joint  service  acquisition  programs  often  involve  highly 
interdependent,  complex  system-of-systems  (SoS)  that  usually  end  in  a  bilateral 
monopoly  contractual  setting.10  In  this  case,  assuming  no  specific  assets  are 
required,  the  outcome  depends  on  the  degree  of  contractual  ambiguity  governing  the 
transaction,  as  well  as  on  any  administrative  and  enforcement  costs  involved. 
However,  as  complexity,  uncertainty,  and  opportunism  due  to  specific  investments 
increase,  so  does  the  risk  of  hold-up  and  so  do  the  coordination  and  motivation 
(transaction)  costs  required  to  measure,  monitor,  and  govern  both  the  internal 
relationships  among  the  Components  and  the  external  relationship  with  the 
contractor.  These  less-favorable  characteristics  of  joint  SoS  programs  can 
discourage  productive  efforts  and  investments  in  both  internal  Component 
relationships  and  external  contractor  relationships,  thus  contributing  to  more  serious 
cost  overruns,  schedule  breaches,  and  performance  shortfalls. 


10  Many  factors  conspire  to  create  this  bi-lateral  monopoly.  On  the  buyer  side,  monopsony  power 
partly  derives  from  the  fact  the  military  value  of  most  systems  depends  solely  on  their  performance 
relative  to  the  systems  of  adversaries.  This  is  specific  to  a  country  and  the  defense  environment  it 
faces  at  a  particular  point  in  time,  effectively  making  it  the  sole  buyer  of  a  highly  differentiated 
product.  The  appearance  of  a  superior  alternative  results  in  what  might  be  termed  military 
obsolescence.  Response  to  new  threats  can  require  redesign  during  development,  and  modifications 
during  the  system’s  operational  life.  This  cause-and-effect  relationship  conspires  to  reduce  the 
number  of  buyers  of  a  particular  weapon  system,  since  these  weapon  systems  are  often  evolving 
products  (spiral  acquisition).  Thus,  in  addition  to  technical  uncertainty,  there  is  a  significant  degree  of 
demand  uncertainty. 
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6.  Coordination  Costs:  Interdependency  yields 
Complexity  and  Uncertainty 


Interdependency  is  typically  defined  as  the  degree  to  which  the  performance 
of  one  activity  (or  system)  relies  on  an  external  activity  (or  system)  for  its  success 
(Thompson,  1967).  Under  conditions  in  which  organizations  are  allowed  to  seek  the 
most  efficient  path  to  task  accomplishment,  interdependent  relationships  will  be 
established  as  long  as  the  benefits  exceed  the  costs.  Private  entities  typically  make 
technology  investments  and  seek  interdependencies  to  achieve  the  benefits  of 
synergy  and  economies  of  scale  based  on  measurable  effects  on  the  “bottom  line.” 
In  contrast,  government  agencies  are  often  guided  and  constrained  by  legislative 
requirements  for  cross-organizational  integration  to  establish  interdependent 
relationships.  Consider,  for  example,  the  increased  emphasis  on  “jointness”  since 
the  Goldwater  Nichols  Department  of  Defense  Reorganization  Act  of  1986.  The 
intent  of  this  legislation  was  clearly  to  increase  interoperability,  and  hence 
interdependency,  in  the  DoD.  Combined  with  recent  reforms  that  reinforce  joint 
solutions  to  defense  capability  needs  (such  as  JCIDS),  simultaneously  encouraging 
SoS,  the  result  is  increased  interdependency  reflected  in  increased  complexity  and 
uncertainty,  and  correspondingly  higher  coordination  costs.11 

For  most  systems,  interoperability  is  pursued  as  a  means  to  leverage  the 
collective  assets  of  various  organizations  located  at  different  points  along  the  value 


11  Interdependent  activities  are  not  new  to  the  DoD  or  to  government  in  general.  However,  what  is 
new  is  the  scale  in  which  interdependent  actions  are  applied.  Prior  to  the  information  technology 
revolution  of  the  1990s,  spatial  and  temporal  distances  tended  to  impede  communication  and  the 
sharing  of  information  among  partners.  Hence,  tightly  coupled  activities  were  generally  restricted  to 
small  groups  of  geographically  co-located  groups  where  coordination  costs  could  be  minimized.  The 
advent  of  advanced  information  and  communication  technologies  eroded  many  of  the  spatial  and 
temporal  barriers  that  once  thwarted  collaboration.  The  potential  benefits  of  information-sharing 
enabled  by  interoperability  were  then  quickly  realized  and  became  a  major  thrust  for  many 
organizations,  the  DoD  included.  Network  externalities  were  increasingly  recognized — the  value  of 
the  network  to  individual  participants  increased  with  the  number  of  participants  connected  to  the 
network. 
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chain.  For  example,  in  the  command  and  control  (C2)  process,  military  operations 
benefit  when  commanders  can  seek,  synthesize,  and  disseminate  several  types  of 
information  derived  from  different  organizations.  Experts  in  a  variety  of  areas  must 
collaborate  during  the  C2  process  to  effectively  create  and  execute  battle  plans. 
These  experts  may  come  from  different  disciplines  (or  specialties),  different 
branches  of  the  military,  or  even  different  countries.  In  short,  interoperable  systems 
promote  interdependent  actions.  In  turn,  increased  interdependency  is  reflected  in 
increased  complexity  and  uncertainty  and  higher  coordination  costs. 

Complexity  is  a  key  component  of  transaction  costs.  When  advanced 
(immature)  technologies  are  combined  with  systems  integration  challenges  across 
diverse  organizations  in  the  scale  of  joint  service  SoS  programs,  the  resulting 
complexity  leads  to  higher  coordination  costs.  Marshall  and  Meckling  (1962)  were 
among  the  first  to  discover  that  variability  in  the  size  of  cost-estimating  errors  in 
defense  contracts  could  at  least  partly  be  attributed  to  technological  complexity,  with 
larger  errors  associated  with  greater  technological  advances  sought  in  different 
systems.12 

Uncertainty  is  another  key  component  of  transaction  costs.  The  reconciliation 
of  competing  requirements  of  different  players  in  joint  programs  can  lead  to  design 
changes  and  implementation  challenges  (demand  uncertainty).  Similarly,  the  “free 
rider  problem”  (in  which  none  of  the  players  want  to  sacrifice  individual  budgets  to 
cover  costs  that  might  benefit  others),  combined  with  changes  in  Congressional 
priorities  (political  uncertainty),  can  lead  to  funding  instability  (budget  uncertainty). 


12  Adler  (1995)  examines  the  complex  and  interdependent  relationship  between  product  design  and 
manufacturing,  describing  four  possible  governance  mechanisms  to  improve  coordination  (e.g., 
standards,  schedules,  mutual  adaptation  and  teams).  McNaugher  (1989)  provides  evidence  that 
costs  rise  rapidly  with  system  complexity,  as  does  the  variance  of  costs  around  expected  costs  (p. 
128).  Consider  the  increase  in  complexity  in  the  US  Navy’s  new  Littoral  Combat  Ship  (LCS)  class, 
which  has  experienced  a  128%  cost  growth  and  which  was  designed  to  avoid  costs  through  smaller 
crews  (substituting  capital  for  labor).  It  is  unclear  whether  the  substantial  increase  in  transaction  costs 
and  scope  for  opportunism  introduced  with  the  increased  complexity  (capital  investment  in  complex 
onboard  systems)  justifies  the  anticipated  labor-cost  savings. 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


- 18  - 


Besides  demand,  political,  and  budget  uncertainty,  joint  programs  face 
measurement  uncertainty,  technological  uncertainty,  supplier  performance 
uncertainty,  etc.13 

An  interesting  avenue  for  future  research  would  be  to  investigate  if  increased 
emphasis  on  “jointness”  since  the  Goldwater  Nichols  Department  of  Defense 
Reorganization  Act  of  1986  or  the  advent  of  the  JCIDS  in  2003  has  increased  the 
complexity  and  uncertainty  of  joint  programs  relative  to  others  and,  consequently, 
raised  coordination  and  motivation  costs.  The  possibility  exists  that  other 
characteristics  inherent  in  many  joint  programs  might  offset  these  higher  transaction 
costs. 


On  the  one  hand,  a  joint  program  manager  that  manages  a  program  that  is 
technologically  mature,  that  does  not  require  strict  military  specifications,  in  which 
funding  and  requirements  are  relatively  stable,  and  in  which  a  contestable  market 
exists  for  the  product  or  service,  may  in  fact  experience  lower  transaction  costs.  On 
the  other  hand,  if  the  program  is  facing  immature  technologies,  rigid  specifications, 
funding  and  requirements  instability,  and  monopolistic  suppliers,  joint  program 
managers  and  other  key  decision-makers  should  recognize  the  potential  for  high 
transaction  costs  and  opportunistic  behavior. 


13  A  dynamic  programming  model  by  Womer  &  Terasawa  (1989)  finds  that  under  demand 
uncertainty,  a  rational  defense  contractor  must  prepare  for  various  contingencies,  and  will,  for 
example,  restrict  investments  in  specific  assets,  which  drives  costs  higher  than  they  would  be  if 
demand  were  certain.  This  tends  to  increase  information  and  contracting  costs,  and  as  the  authors 
demonstrate,  threatens  investments  in  specific  assets.  The  authors  show  that  the  higher  the 
probability  the  contract  will  be  canceled,  the  less  the  contractor  will  invest  in  capital  equipment 
(relation-specific  investments),  which  results  in  relatively  more  labor-intensive  production  and  raises 
costs.  Thus,  demand  uncertainty  increases  contracting  costs  and  also  raises  issues  related  to  asset 
specificity.  Under  demand  uncertainty,  the  rational  contractor  will  restrict  investments  in  specific 
assets  (such  as  capital  tooling  or  specialized  expertise),  unless  they  are  reasonably  sure  to  recover 
these  costs  via  overhead.  Allowing  “Facilities  Capital  Cost  of  Money”  to  be  an  expense  is  a 
mechanism  by  which  the  government  reduces  the  risk  to  the  contractor  by  allowing  recovery  of  costs 
associated  with  specific  capital  assets  (buildings,  tooling,  etc.)  through  overhead.  Critics  argue  this 
encourages  defense  contractors  to  over-capitalize,  which  increases  costs  DoD-wide. 
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TCE  and  Cost  Estimating 


The  higher  the  transaction  costs,  the  greater  the  economic  production  cost 
efficiencies  need  to  offset  them.  As  recently  emphasized  by  Melese  et  al.  (2007),  if 
initial  cost  estimates  focus  exclusively  on  economic  production  costs  and  ignore 
transaction  costs,  the  result  can  be  systematic  cost  overruns. 

According  to  a  recent  RAND  study  of  major  acquisition  programs,  the  average 
cost  overruns  of  weapon  systems  in  the  development  phase  ranged  from  16%  to 
26%.  Procurement  cost  growth  over  initial  estimates  averaged  between  16%  and 
65%,  while  total  weapon  program  cost  overruns  averaged  from  20%  to  54%  (Arena, 
Leonard,  Murray,  &  Younossi,  2006). 

Since  the  official  “acquisition  program  baseline”  (APB)  estimates  for  these 
programs  reflect  the  best  current  understanding  of  the  cost,  schedule,  and 
performance  objectives  at  the  time  the  baseline  is  established  (typically  at  Milestone 
B  decisions),  Congressional  funding  of  these  programs  represents  an  implicit 
contract  with  the  Executive  Branch.  When  incomplete  or  unrealistic  cost  estimates 
lead  to  significant  cost  and  schedule  overruns  relative  to  expectations  established  in 
the  APB,  administrative  sanctions,  such  as  statutory  (Nunn-McCurdy14)  unit  cost 


14  Since  the  law  was  enacted  in  1982,  Title  10  USC  Section  2433,  a  “Nunn-McCurdy”  unit  cost  breach 
occurs  when  a  major  defense  acquisition  program  experiences  an  increase  of  at  least  15%  in 
program  acquisition  unit  cost  or  average  procurement  unit  cost  above  the  unit  costs  in  the  acquisition 
program  baseline.  Through  2006,  the  DoD  had  the  ability  to  administratively  change  the  acquisition 
program  baseline  for  the  purposes  of  unit  cost  reporting  and  so  was  able  to  reduce  the  number  of 
apparent  Nunn-McCurdy  breaches,  despite  apparent  cost  growth  that  would  otherwise  trigger  the 
Nunn-McCurdy  sanctions.  In  2006,  the  Nunn-McCurdy  law  was  amended.  The  FY  2006  National 
Defense  Authorization  Act  ( NDAA )  severely  restricted  the  DoD’s  ability  to  change  unit  cost  reporting 
criteria.  As  a  result,  Nunn-McCurdy  unit  cost  breaches  are  incurred  at  15%  (“Significant”),  and  25% 
(“Critical”)  above  the  current  unit  cost  threshold.  Additionally,  a  “Significant”  Nunn-McCurdy  breach  is 
incurred  at  30%  of  the  Milestone  B  unit  cost  threshold,  and  a  “Critical”  Nunn-McCurdy  breach  is 
declared  at  50%  growth  above  the  Milestone  B  unit  cost  threshold.  Thus,  the  ability  of  the  DoD  to 
mask  unit  cost  growth  through  changes  to  unit  cost  thresholds  is  restricted. 

The  sanctions  imposed  by  Congress  on  programs  breaching  Nunn-McCurdy  criteria  are  noteworthy: 
For  “Significant”  breaches,  the  service  Secretary  must  notify  Congress  within  45  days  of  the  report 
(normally  program  deviation  report)  upon  which  the  determination  is  based  (normally  a  program 
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breaches,  can  be  triggered.  In  turn,  these  breaches  can  dramatically  impact  program 
execution;  they  also  jeopardize  relations  between  the  Legislative  and  Executive 
Branches  of  government. 

A  study  conducted  by  the  DoD  in  2007  to  develop  a  business  case  for 
improving  system  cost  estimating  in  the  DoD  (Brown,  Flowe,  &  Hamel,  2007a) 
examined  the  cost  growth  of  all  major  defense  acquisition  programs  (MDAPs)  from 
1995  to  2005  to  determine  the  source  and  relative  magnitude  of  cost  growth  and 
schedule  breaches.  See  Figure  2. 


deviation  report  initiated  when  the  Program  Manager  becomes  aware  of  the  breach).  The  program 
must  submit  a  Selected  Acquisition  Report  (SAR)  with  the  required  unit  cost  breach  information. 


For  “Critical”  breaches,  the  Defense  Secretary  (usually  delegated  to  the  Under  Secretary  of  Defense 
for  Acquisition,  Technology,  and  Logistics)  must  certify  to  Congress  within  60  days  of  notification  that: 

1 )  the  program  is  essential  to  national  security, 

2)  there  is  no  alternative  which  can  provide  equal  capability  at  less  cost, 

3)  the  updated  estimates  of  unit  cost  (calculated  independently  by  the  OSD  Cost  Analysis 
Improvement  Group)  are  reasonable,  and 

4)  the  management  structure  is  adequate  to  control  unit  cost  going  forward. 

Failure  to  certify  within  the  60-day  timeframe  will  result  in  suspension  of  obligations  for  major 
contracts  until  30  days  of  continuous  session  of  Congress,  beginning  from  the  date  of  receipt  of 
SAR/Certification. 
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Ten  Year  Cost  Change  by  Category 
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Figure  2.  Cost  Growth  by  Category  for  All  MDAPs  1995-2005 

(Brown,  Flowe,  &  Hamel,  2007a) 

As  shown  in  Figure  2,  the  largest  source  of  cost  growth  as  reported  in  the 
Selected  Acquisition  Reports  (SAR)  is  “Estimation.”  This  indicates  that  estimates 
made  in  establishing  MDAP  acquisition  program  baselines  were  often  in  error,  and 
thus  program  costs  appeared  to  grow  despite  very  little  change  in  the  objective 
content  of  the  program.  This  “Estimation”  error  accounted  for  approximately  $201 
billion  in  apparent  cost  growth  over  the  10-year  period  examined  across  all  major 
programs. 

Figure  3  is  a  related  analysis  that  examines  the  quantity  and  sources  of 
breaches  for  108  MDAPs  over  the  same  10-year  period  (1995-2005).  With  respect 
to  the  Acquisition  Program  Baseline  (APB),  schedule  breaches  were  the  most 
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common;  occurring  244  times  over  the  10-year  period  examined,  suggesting  that 
multiple  schedule  breaches  is  a  relatively  common  occurrence  in  many  programs. 


Breach  Incidence  by  Type 
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Figure  3.  Breaches  Reported  by  SAR  All  MDAPs  1995-2005 
(Brown,  Flowe,  &  Hamel,  2007a) 

Development  and  procurement  cost  breaches  occur  with  less  frequency  than  schedule 
breaches  but  still  occur  sufficiently  frequently  that  on  average,  each  MDAP  can  expect  to  have 
one  of  each.  Program  Acquisition  Unit  Cost  (PAUC)  breaches  occur  nearly  as  frequently  as 
schedule  breaches,  indicating  that  the  confluence  of  development  and  procurement  cost 
increases  conspire  to  increase  unit  costs  at  least  10%  above  that  established  by  the  APB. 
Nunn-McCurdy  breaches  are  notably  less  common,  suggesting  that  only  about  20%  of 
programs  that  breach  their  APB  unit  costs  will  grow  substantially  beyond  that.  Overall,  the 
raw  quantity  of  breaches  indicate  that  expectations  regarding  costs  and  schedules  are  usually 
unmet,  to  the  extent  that  for  the  108  MDAPs  examined,  each  breached  on  average  more  than 
twice  for  schedule  and  unit  cost  and  at  least  once  for  development  cost.15 


15  This  is  not  to  imply  that  every  program  was  equally  troubled.  A  subset  of  particularly  troubled 
programs,  approximately  30%  of  the  total,  breached  in  some  way  every  year  they  reported. 


NPS 


prMVW^TIA  PER 

MI1I1IJ1  - - / 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-24  - 


8 


Cost  and  Schedule  Breaches 


We  investigated  cost  and  schedule  breaches  in  a  subset  of  MDAPs  that 
included  a  sample  of  84  programs,  divided  into  “joint  service”  and  “traditional”  (single 
service)  acquisition  programs,  and  “single  system”  and  “system-of-systems”  (SoS) 
programs.  The  84  DoD  weapon  system  programs  were  initially  examined  by  Brown, 
Flowe,  and  Hamel  (2007b)  and  were  identified  as  being  either  “single  system”  or 
“system-of-systems.”  The  “system-of-systems”  designator  was  used  as  a  proxy  for 
complexity. 

While  it  seems  reasonable  to  assume  that  programs  designated  as  “system- 
of-systems”  will  be  more  complex,  it  is  not  necessarily  true  that  such  programs  will 
be  joint  programs  involving  more  than  one  service.  For  example,  an  aircraft  carrier 
is  certainly  a  “system-of-systems”  but  is  managed  by  a  single  service,  the  Navy. 
Another  example  would  be  the  F-22,  arguably  a  “system-of-systems”  and  certainly 
complex,  but  still  only  managed  by  a  single  service,  in  this  case  the  Air  Force.  On 
the  other  hand,  there  may  be  “single  systems”  that  are  considered  joint  programs 
because  they  are  either  managed  or  procured  by  more  than  one  service. 

SoS  programs  are  defined  as  independent  systems  that  are  integrated  into  a 
larger  system  to  provide  unique  capabilities.  The  identification  of  a  program  as  SoS 
can  be  seen  as  a  proxy  for  complexity.  Joint  programs  are  systems  that  involve 
more  than  one  service  during  any  phase  of  the  program  (i.e.,  development, 
production,  operations)  and  can  be  seen  as  proxy  for  coordination  costs.  Using  this 
distinction  between  “system-of-systems”  and  “joint,”  we  assigned  a  “joint  service” 
designator  to  the  84  DoD  weapon  systems  originally  used  by  Brown,  Flowe,  and 
Hamel  (2007b).  Table  1  shows  the  programs  divided  into  “single  service”  and  “joint 
service”  categories. 
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Single  Service  (n  =  58) 

Joint  Service  (n  =  26) 

PATRIOT  PAC-3 

H-1  UPGRADES  (4BW/4BN) 

CVN  68 

NAVSTAR  GPS 

EELV 

AMRAAM 

TRIDENT  II  MISSILE 

MH-60R 

ARH 

F-35  (JSF) 

DDG  51 

V-22 

DDG  1000  (DD(X)) 

ATIRCM/CMWS 

C-17A 

GBS 

SBIRS  HIGH 

NPOESS 

C-130J 

AEHF 

CVN  21 

MH-60S 

T-45TS 

JTRS  JOINT  WAVEFORM 

SSDS 

MP  RTIP 

MINUTEMAN  III  PRP 

WGS 

ADS  (AN/WQR-3) 

BLACK  HAWK  UPGRADE  (UH-60M) 

GLOBAL  HAWK  (RQ-4A/B) 

JTRS  GMR  (CLUSTER  1) 

GMLRS 

BMDS 

F-22A 

AGM-88E  AARGM 

E-2C  REPRODUCTION 

JTRS  HMS  (CLUSTER  5) 

CH-47F 

JDAM 

JAVELIN 

PATRIOT/MEADS  CAP 

LAND  WARRIOR 

MIDS 

TACTICAL  TOMAHAWK 

JASSM 

FBCB2 

JPATS 

WIN  -T 

AIM-9X 

C-130  AMP 

JSOW 

STRYKER 

FCS 

MINUTEMAN  III  GRP 

C-5  RERP 

AESA 

LHA  REPLACEMENT 

SSGN 

MUOS 

SDB  1 

E-2D  AHE 

COBRA  JUDY  REPLACEMENT 

EXCALIBUR 

HIMARS 

JLENS 

LCS 

B-2  RMP 

EA-18G 
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ASDS 

SM-6 

VH-71  (VXX) 

MPS 

EFV 

SSN  774  (VIRGINIA  CLASS) 

NAS 

LPD  17 

F/A-18E/F 

CEC 

T-AKE 

BRADLEY  UPGRADE 

MCS 

FMTV 

LONGBOW  APACHE 

Table  1.  Single  Service  and  Joint  Service  Programs 


a.  Methodology 

A  set  of  84  DoD  weapon  system  programs  first  divided  into  “single  service” 
and  “joint  service”  programs  by  Brown,  Flowe,  and  Hamel  (2007b)  were  further 
divided  into  either  “single  system”  or  “system-of-systems”  (SoS)  programs.  (Data 
available  upon  request.)  Our  tests  focused  on  four  categories  of  breaches: 

Schedule,  RDT&E,  Program  Acquisition  Unit  Cost  (PAUC),  and  Average 
Procurement  Unit  Cost  (APUC)  defined  as  follows: 

■  Schedule  Breach:  When  schedule  exceeds  most  recent  Acquisition 
Program  Baseline  (APB)  schedule  estimate  by  6  months 

■  RDT&E  Breach:  When  the  program’s  research,  development,  test 
and  evaluation  costs  exceed  15%  of  the  baseline  threshold. 

■  PAUC  Breach:  When  the  program  acquisition  unit  cost  exceeds  the 
most  recent  APB  threshold  by  15%.  This  is  a  Congressionally 
reportable  breach. 

■  APUC  Breach:  When  the  average  procurement  unit  cost  exceeds  the 
most  recent  APB  threshold  by  15%.  This  is  a  Congressionally 
reportable  breach. 
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Our  hypothesis  is  that  SoS  programs  are  more  complex  than  single  system 
programs,  so  they  will  be  more  likely  to  experience  cost  and  schedule  breaches. 
Likewise,  Joint  programs  require  more  coordination  than  traditional  (single  service) 
programs  and  are  also  more  likely  to  experience  cost  and  schedule  breaches. 
Schedule  and  cost  breaches  may  be  evidence  of  hidden  transaction  costs. 

To  investigate  the  differences  between  the  data  groupings  described  above, 
we  ran  an  analysis  of  variance  using  “jointness”  as  the  grouping  variable  instead  of 
“system-of-systems.”  Because  the  subdivided  samples  were  not  normally 
distributed,  we  also  ran  a  Kruskal-Wallis  (H)  test  to  determine  if  there  was  a 
significant  difference  in  the  mean  ranks  of  the  groups,  testing  single  service  vs.  joint 
service  and  single  systems  vs.  system-of-systems  (SoS).  The  H  test  is  particularly 
robust  as  it  does  not  make  any  assumptions  about  the  underlying  distribution  of  the 
samples.  We  also  considered  whether  there  might  be  a  difference  in  average 
number  of  breaches  between  development  and  production  programs.  We  used  a 
Mann-Whitney  (U)  test  (which  does  not  make  any  assumptions  about  the  underlying 
distribution  of  the  samples)  to  test  our  hypothesis.  We  looked  at  the  difference  in 
mean  ranks  for  schedule  breaches,  cost  breaches  (including  RDT&E,  procurement, 
PAUC  and  APUC)  and  total  breaches. 

b.  Results 

Table  2  compares  the  results  of  the  tests  based  on  the  two  different 
groupings.  While  it  is  clear,  as  noted  by  Brown,  Flowe,  and  Hamel  (2007b)  that 
there  is  a  significant  difference  (at  the  .05  level)  in  the  average  number  of  schedule 
breaches  and  RDT&E  cost  breaches  between  “single  system”  and  “system-of- 
systems”  programs,  there  is  no  significant  difference  between  “single  service”  and 
“joint  service”  for  any  of  the  breach  categories  examined. 
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Variable 

Single 

System 

System 

of 

Systems 

Single 

Service 

Joint 

Service 

Mean 

(n=39) 

Mean 

(n=45) 

p-value 

Mean 

(n=58) 

Mean 

(n=26) 

p-value 

Schedule 

breaches 

4.49 

8.60 

0.0229 

5.71 

8.88 

0.1058 

RDT&E 

breaches 

1.64 

5.87 

0.0010 

3.48 

4.85 

0.3404 

PAUC 

breaches 

2.36 

4.24 

0.0693 

3.17 

3.81 

0.5739 

APUC 

breaches 

1.10 

1.82 

0.1678 

1.48 

1.50 

0.9757 

Table  2.  Analysis  of  Variance 


This  data  suggests  that  while  the  “system-of-systems”  designator  may  be  a 
good  proxy  for  complexity,  which  in  turn  may  lead  to  schedule  and  cost  breaches, 
“jointness”  does  not  appear  to  make  a  difference  when  it  comes  to  program 
breaches.  This  is  an  unexpected  result,  since  we  reasoned  that  joint  programs  are 
more  likely  to  experience  higher  transaction  costs  and  therefore  might  experience 
more  cost  breaches  if  their  cost  estimation  models  do  not  include  variables  to 
account  for  those  transaction  costs.  The  data  seems  to  suggest  that  complexity  may 
be  the  overriding  influence  on  schedule  and  cost  breaches  regardless  of  whether  a 
program  is  managed  by  a  single  service  or  multiple  (joint)  services. 

The  results  of  the  H  test  are  reported  in  Table  3.  Again  we  see  that 
complexity  (as  measured  by  “system-of-systems”)  has  a  significant  influence  on  the 
mean  ranks  for  schedule  and  RDT&E  breaches.  We  also  see  that  there  is  a 
significant  difference  between  mean  ranks  for  schedule  breaches  between  “single 
service”  and  “joint  service”  programs  but  not  for  cost  breaches.  As  Brown,  Flowe, 
and  Hamel  (2007b)  noted,  joint  programs  have  more  participants,  divergent 
requirements,  diffuse  authority  and  complex  management  structures  all  of  which 
might  contribute  to  schedule  breaches.  It  appears,  however,  that  those  conditions 
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do  not  necessarily  lead  to  cost  breaches.  Again,  we  may  conclude  that  complexity  is 
more  important  to  cost  breaches  than  “jointness.” 


Variable 

Single  System 

System  of 
Systems 

Single  Service 

Joint  Service 

Mean  Rank 

(n=39) 

Mean  Rank 
(n=45) 

p-value 

Mean  Rank 

(n=58) 

Mean  Rank 
(n=26) 

p-value 

Schedule  breaches 

34.36 

49.56 

0.0039 

38.56 

51.31 

0.0246 

RDT&E  breaches 

32.44 

51.22 

0.0002 

39.34 

49.56 

0.0631 

PAUC  breaches 

39.14 

45.41 

0.2244 

41.70 

44.29 

0.6418 

APUC  breaches 

39.60 

45.01 

0.2633 

42.09 

43.42 

0.7977 

Table  3.  Kruskal-Wallis  (H)  Test 


Overall,  the  results  are  consistent  but  somewhat  weaker  considering  the 
difference  in  the  average  number  of  breaches  between  development  and  production 
programs  using  the  U  test  as  shown  in  Table  4.  The  results  show  a  significant 
difference  between  the  mean  rank  of  schedule,  cost  and  total  breaches  in 
development  programs  based  on  complexity  (as  measured  by  “system-of-systems”). 
In  each  case,  the  mean  rank  is  higher  for  “system-of-systems”  programs  than  for 
“single  system”  programs.  We  also  see  a  significant  difference  in  mean  ranks  of 
cost  and  total  breaches  in  production  programs  based  on  complexity.  Again,  the 
mean  ranks  are  higher  for  “system-of-systems”  programs  than  for  “single  system” 
programs.  But  there  is  no  significant  difference  in  mean  ranks  of  either  development 
or  production  program  breaches  between  “single  service”  and  “joint  service.”  This 
reinforces  our  earlier  observations  that  complexity  (as  measured  by  “system-of- 
systems”)  appears  to  be  a  stronger  indicator  of  possible  schedule  and  cost  breaches 
than  “jointness.” 
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Variable 

Single  System 

vs. 

System-of-systems 

Single  Service 

vs. 

Joint  Service 

Development 

Production 

Development 

Production 

n=48 

n=36 

n=48 

n=36 

Schedule 

breaches 

0.0104 

0.1558 

0.0624 

0.2126 

Cost  breaches 

0.0414 

0.0225 

0.2603 

0.5703 

Total 

0.0276 

0.0289 

0.1632 

0.4497 

Table  4.  Mann-Whitney  (U)  test 


These  findings  should  be  interpreted  with  a  note  of  caution.  For  example,  the 
limited  sample,  the  method  for  categorizing  the  degree  of  interdependence  or 
“jointness”  and  SoS,  and  the  failure  to  include  and  control  for  other  important  factors 
may  be  significant.  Though  preliminary,  these  results  offer  evidence  to  support 
further  investigations  on  the  role  of  “jointness”  and  SoS  in  program  acquisition. 

c.  Discussion 

The  results  suggest  there  is  a  statistically  significant  higher  risk  of  cost  and 
schedule  breaches  in  SoS  programs  than  in  “single  system”  acquisition  programs. 
Interestingly,  while  “joint  service”  programs  in  general  have  a  weak  statistically 
significant  greater  risk  of  schedule  (and  RDT&E  cost)  breaches  than  “traditional” 
(single  service)  programs,  there  is  no  significant  difference  between  the  two  in  terms 
of  Program  Acquisition  Unit  Cost  (PAUC)  breaches  or  Average  Procurement  Unit 
Cost  (APUC)  breaches. 

Based  on  our  sample,  SoS  programs  tend  to  take  relatively  longer  and  cost 
more  than  “single  system”  acquisitions.  This  preliminary  empirical  evidence  of  cost 
and  schedule  breaches  suggests  initial  cost  (schedule)  estimates  of  SoS  programs 
may  not  be  adequately  capturing  transaction  costs.  In  fact,  since  production  cost 
breaches  are  significantly  greater,  the  transaction  costs  experienced  by  SoS 
programs  may  be  overwhelming  any  potential  economic  production  cost  savings 
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(Melese  et  al.,  2007).  If  this  is  indeed  the  case,  the  anticipated  warfighting  benefits  of 
SoS  solutions  need  to  be  sufficiently  large  to  compensate  for  the  extra  costs  and 
schedule  delays  experienced  by  these  programs. 

Meanwhile,  it  appears  that  while  “jointness”  contributes  to  schedule  overruns, 
it  only  weakly  contributes  to  development  cost  overruns,  and  by  itself,  does  not 
explain  production  cost  overruns.  While  our  results  suggest  joint  programs  tend  to 
breach  their  schedules  relatively  more  often  than  “single  service”  programs  (“every 
event  in  a  joint  program  takes  longer  [...]  extra  time  needs  to  be  included  in  the 
program  schedule”  (DAU,  2004,  p.  20)),  “joint  service”  programs  in  our  sample  only 
experienced  a  few  more  cost  breaches  than  “single  service”  programs  in  the  early 
development  (RDT&E)  stages.  There  does  not  appear  to  be  any  statistical  difference 
in  production  cost  breaches  (PAUC  or  APUC)  between  “joint  service”  and  “single 
service”  programs. 

One  explanation  for  this  is  that  the  “joint  service”  programs  in  the  sample 
encompass  a  spectrum  that  includes  both  “single  systems”  and  SoS,  as  well  as  the 
complexity  and  uncertainty  associated  with  SoS  is  so  significant  as  to  overwhelm 
any  additional  complexity  and  uncertainty  that  might  be  experienced  by  joint  service 
programs.  Another  possibility  is  that  built-in  checks  and  balances  tend  to  offset  the 
extra  transaction  costs  of  joint  service  programs. 

For  instance,  “joint  programs  require  special  attention  to  multi-service  funding 
requirements  and  to  acquiring  the  right  mix  of  joint  expertise  for  the  source  selection 
process”;  indeed,  “full  consultation  and  coordination  with  the  participating 
components”  is  required  (DAU,  2004,  pp.  12,  21).  The  ultimate  outcome  may  be  to 
help  anticipate  and  mitigate  the  extra  transaction  costs  and  to  avoid  requirements 
creep  in  production  stages  of  the  program.  Guidance  for  joint  program  managers  is 
designed  to  inhibit  any  changes  in  scope,  stating  that  “substantive  changes  to  [...] 
program  documentation,  such  as  the  acquisition  strategy  or  the  contract  [need  to  be] 
negotiated  with  the  participating  Components  prior  to  making  changes”  (p.  12). 
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With  weapon  system  investments  expected  to  capture  a  significant,  and 
perhaps  growing,  share  of  defense  budgets,  unprecedented  attention  has  been 
devoted  to  clarifying  the  determinants  of  risk,  failure,  and  success  in  the  joint  arena 
(Pracchia,  2004).  The  Defense  Department’s  apparent  inability  to  avert  or  even 
predict  adverse  program  outcomes  such  as  cost  and  schedule  breaches  is  not  only 
a  source  of  external  criticism  (GAO,  2006)  and  internal  attention  (Krieg,  2005),  it  has 
undermined  confidence  in  the  time-honored  practices  of  program  management  and 
oversight. 

To  date,  there  is  significant  debate  regarding  the  factors  that  influence  the 
outcomes  of  programs.  Thus,  the  search  for  root  causes  and  potential  solutions  of 
program  cost  growth,  schedule  delay,  and  capability  shortfall  have  received 
increased  attention.  To  help  explain  potential  pitfalls  associated  with  joint  programs, 
this  study  leverages  “Transaction  Cost  Economics”  (TCE),  which  has  recently  been 
applied  to  generate  new  insights  into  defense  cost  overruns  (Melese  et  al.,  2007). 
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Conclusion 


Transaction  Cost  Economics  (TCE)  suggests  investigating  the  specific 
characteristics  of  transactions  that  make  up  joint  service  programs  (like  SoS)  could 
help  anticipate  (and  perhaps  mitigate)  cost  and  schedule  breaches.  Since  increases 
in  interoperability/interdependence  tend  to  increase  complexity  and  uncertainty,  and 
complexity  and  uncertainty  increase  coordination  and  motivation  costs,  it  is  likely 
that  cost  and  schedule  breaches  partly  depend  on  decisions  regarding  the  extent  of 
bundling  or  unbundling  of  the  many  interdependent  parts  of  joint  systems,16  and  on 
the  particular  phase  of  development  or  production  of  those  weapon  systems. 

Historically,  fixed-price  contracts  are  usually  prescribed  in  later  stages  of 
product  development  when  complexity  and  uncertainty  have  been  resolved,  and  the 
contract  is  complete.  In  contrast,  cost-reimbursement  contracts  are  usually 
prescribed  in  earlier  stages  of  product  development  when  complexity  and 
uncertainty  have  not  been  resolved,  and  the  contract  is  incomplete.17  Today,  cost- 
reimbursement  contracts  are  phased  out  and  fixed-price  contracts  phased  in,  as 
complexity  and  uncertainty  issues  are  resolved.  However,  complexity  and 


16  For  instance,  an  extremely  complex  interoperable  system  is  envisioned  for  the  Coast  Guard’s  new 
“integrated  deepwater  system  program.”  The  system  is  intended  to  include  cutters  and  small  boats,  a 
new  fleet  of  fixed-wing  aircraft,  a  combination  of  new  and  upgraded  helicopters,  and  land-  and  cutter- 
based  unmanned  aerial  vehicles  (UAVs) — all  linked  with  Command,  Control,  Communications  and 
Computers,  Intelligence,  Surveillance  and  Reconnaissance  (C4ISR)  systems  and  supported  by 
integrated  logistics.  According  to  a  recent  Coast  Guard  press  release  (2007,  June  25),  “Deepwater  is 
a  25-year,  $24  billion  effort  that  will  produce  more  than  91  new  cutters;  195  new  aircraft”  and  C4ISR 
equipment.  The  lead  systems  integrator  is  a  joint  venture  between  Lockheed  Martin  and  Northrop 
Grumman  that  has  recently  been  in  the  news  for  major  cost  overruns,  schedule  slippages,  and  quality 
issues — the  latter  involving  several  modified  ships  that  were  determined  un-seaworthy. 

17“Complexity”  and  “technological  uncertainty”  (as  opposed  to  “demand  uncertainty”)  are  usually 
correlated.  Ignorance  about  what  buyers  want  and  what  contractors  can  do  result  in  large  up-front 
search  and  information  or  Research  and  Development  (R&D)  costs.  R&D  is  similar  to  a  real  option  in 
the  sense  that  real  options  models  are  learning  models(Kogut  &  Kulatilaka,  2001).  The  problem  that 
gives  rise  to  high  transaction  costs  in  the  case  of  complex  weapon  systems  is  that  this  characteristic 
of  the  transaction  leads  to  market  failure  (missing  markets).  From  a  TCE  perspective,  the  classic 
market  failures — natural  monopoly,  negative  externalities,  public  goods,  etc. — have  information 
analogues:  missing  markets,  adverse  selection,  and  moral  hazard. 
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uncertainty  must  be  characterized  in  the  context  of  TCE  for  this  strategy  to  succeed. 
Note  that  while  these  prescribed  contracts  focus  on  the  characteristics  of  complexity 
and  uncertainty,  often  overlooked  is  the  vital  role  of  asset  specificity — a  key 
component  reflected  in  the  motivation  cost  component  of  transaction  costs. 

The  main  strategy  of  reducing  cost  and  schedule  breaches  is  employed  to 
identify  ways  to  cut  coordination  and  motivation  costs.  Specific  recommendations 
include:  a)  reducing  complexity  by  investing  in  a  more  complete  contract  (e.g., 
setting  realistic  baselines — entailing  higher  search  and  information  costs — or  using 
more  mature  technologies,  recently  emergent  in  acquisition  policies;  b)  reducing 
uncertainty  through  multi-year  contracts  (reducing  demand  uncertainty)  or  investing 
in  a  more  complete  contract  (reducing  relationship  uncertainty);  c)  increasing 
measurement  (CAIG)  and  monitoring  (GAO)  of  performance  and  both  production 
and  transaction  costs  to  reduce  information  asymmetries  and  the  associated  risks  of 
moral  hazard  and  adverse  selection;  d)  placing  credible  deterrents  to  bad  behavior 
in  place — such  as  penalty  clauses,  warranties  and  bonding;  e)  using  multi-year 
contracts  to  gather  information  and  to  reward  good  reputations  (Kelman,  1990);  f) 
mitigating  opportunistic  behavior  introduced  by  asset  specificity  through  careful  use 
of  incentives,  proper  bundling  (or  task-partitioning)  of  joint  programs,  and  strategic 
investments  in  government-furnished  equipment  or  government-owned  and 
contractor-operated  assets;  and  finally  g)  increasing  market  contestability  through 
investments  in  real  options  (e.g.,  government-controlled  standby  capacity — credible 
threat  of  vertical  integration;  or  second  sourcing — credible  threat  of  entry).18 


18  Since  full-scale  competition  can  involve  duplicating  high  fixed  costs  and  can  require  significant 
investments  in  specific  assets,  reliance  on  multiple  sources  is  often  prohibitively  costly.  Instead, 
duplicative  supply  has  often  been  a  way  to  hedge  against  technical  uncertainty.  For  example,  in  the 
crash  effort  to  develop  ICBMs,  two  fully  duplicative,  concurrent  programs  were  used  to  cut  lead 
times — the  Atlas  and  Titan  projects.  In  more  limited  attempts  to  secure  competition,  parallel  efforts 
can  be  applied  in  early  stages  of  acquisitions  (such  as  in  systems  development),  with  production  then 
being  allocated  to  a  single  source  (“fly  before  you  buy”).  In  practice,  however,  the  greater  the  degree 
of  asset  specificity  and  fixed  costs  required  in  these  design  competitions,  the  less  effective  they  are; 
indeed,  knowledge  is  not  easily  transferred  and  close  integration  is  needed  between  design  and 
production.  As  witnessed  in  the  Air  Force’s  F-22  program,  GAO  (2006)  reports  200%  cost  overruns; 
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We  believe  any  evaluation  of  joint  service — and  particularly  SoS — acquisition 
programs  would  benefit  from  an  analysis  of  the  characteristics  of  the  corresponding 
bundles  of  transactions  through  the  lens  of  TCE.  An  inspired  effort  to  collect  and 
analyze  data  guided  by  TCE  (as  described  in  this  paper)  could  help  DoD  decision¬ 
makers  anticipate  and  mitigate  cost  and  schedule  breaches  and  avoid  future 
performance  shortfalls. 


the  bulk  of  costs  in  most  programs  occur  in  the  post-design  stages.  The  threat  of  second  sourcing 
(turning  to  another  supplier)  may  be  one  way  to  inject  discipline  into  the  supply  process.  But  if  this 
threat  is  perceived  by  the  seller  to  increase  the  risk  of  opportunistic  behavior  by  the  government 
buyer,  then  this  added  risk  will  raise  the  initial  bids.  Assuming  effective  competition  existed  for  the 
initial  contract,  program  managers  cannot  gain  overall  efficiency  by  using  such  tactics  (Anton  &  Yao, 
1987).  Another  possibility  is  inter-generational  competition  or  to  inject  some  competition  between  a 
new  system  and  its  predecessor.  The  extension  of  existing  systems  provides  some  insurance  against 
delays  in  the  availability  of  the  next-generation  systems. 
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